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1.0  EXECUTIVE  SUMMARY 


Since  2005,  Environmental  Security  Technology  Certification  Program’s  (ESTCP)  Wide  Area 
Assessment  (WAA)  Pilot  Program  has  explored  the  use  of  an  integrated  suite  of  airborne  and 
ground-based  technologies  as  a  means  to  streamline  the  WAA  process.  Light  detection  and 
ranging  (LiDAR)  and  orthophotography,  the  subjects  of  this  demonstration,  were  used  in 
conjunction  with  synthetic  aperture  radar  (SAR),  hyperspectral  sensing,  helicopter-based 
magnetometry,  and  towed-array  magnetometry  and  electromagnetic  induction  (EMI),  along  with 
approaches  to  statistical  modeling  of  transect  design,  in  an  integrated  Geographical  Information 
System  (GlS)-based  analytical  environment. 

The  first  phase  of  the  WAA  Pilot  Program  examined  three  sites — the  Pueblo  Precision  Bombing 
Range  (PBR)  site  near  Pueblo,  Colorado;  the  Kirtland  PBR  site  near  Albuquerque,  New  Mexico; 
and  the  Victorville  DBT  “Y”  site  near  Victorville,  California.  All  three  sites  were  desert 
bombing  ranges  with  little  vegetation  and  few  nonmilitary  land  uses.  The  results  of  the  first 
phase  were  positive.  The  combination  of  technologies  employed  were  successfully  used  to 
locate  munitions  response  sites  (MRS)  and  munitions-related  features  to  correct  the  initial 
conceptual  site  model  (CSM)  and  to  support  further  investigation  by  more  expensive 
technologies  that  directly  detect  munitions  components.  The  combination  of  technologies 
employed  in  the  Pilot  Program  was  cost-effective  and  provided  a  high  degree  of  cross-validation, 
resulting  in  higher  confidence  in  the  overall  results. 

As  a  result,  a  second  phase  was  added  to  the  program,  including  two  additional  sites.  The  first 
site,  Former  Camp  Beale,  is  located  approximately  20  miles  from  Marysville,  California,  just  to 
the  east  of  Beale  Air  Force  Base.  The  site  covers  approximately  18,263  acres  (2,391  hectares) 
and  is  more  complex  than  the  Phase  1  sites,  with  more  vegetation  types,  more  complex 
topography,  and  a  wider  variety  of  land  uses.  The  second  site  was  the  Toussaint  River  site  near 
Lake  Erie,  which  was  primarily  an  underwater  detection  site. 

URS  Corporation  provided  and  analyzed  LiDAR  and  orthophotography  data  for  the  Kirtland, 
Victorville,  and  Former  Camp  Beale  sites.  This  Cost  and  Performance  Report  summarizes  the 
results  of  this  work.  Results  are  described  in  more  detail  in  the  Final  Report  for  the  Kirtland  and 
Victorville  sites  and  the  Final  Report  Addendum  for  the  Former  Camp  Beale  site. 

The  objective  of  the  demonstration  was  to  document  and  validate  the  ability  of  LiDAR  and 
orthophotography  to  contribute  to  the  WAA  process  by: 

•  Identifying  MRS  and  individual  munitions-related  ground  features 

•  Providing  information  about  the  site  and  the  MRS  to  support  future  investigation, 
prioritization,  and  cost  estimation 

•  Providing  information  to  support  regulatory  decisions,  including  decisions  as  to 
requirements  for  further  investigation,  institutional  controls,  or  no  further  action 

•  Describing  the  certainty  associated  with  the  initial  CSM  and  examining  the 
incremental  contributions  of  each  technology  to  improvements  in  that  certainty. 
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An  additional  objective  was  to  develop  information  about  the  factors  that  would  affect  the  cost 
and  performance  of  both  technologies,  including  the  relationship  between  levels  of  effort  and 
confidence  in  conclusions.  Performance  factors  tested  included  orthophoto  and  LiDAR  data 
density,  flight  line  orientation,  and  at  the  Former  Camp  Beale,  preliminary  investigation  of 
vegetation  effects.  Data  artifacts  and  noise  effects  were  observed  and  documented. 

These  objectives  were  met.  LiDAR  and  orthophoto  data  were  successfully  used  to  identify  MRS 
and  munitions-related  features  and  to  verify  and  correct  the  initial  CSM  at  all  three  sites.  ESTCP 
successfully  used  the  LiDAR  and  orthophoto  data  to  plan  subsequent  phases  of  the 
demonstration,  including  the  use  of  helicopter  and  ground-based  magnetometry  and  EMI  sensing 
and  site  reconnaissance.  The  demonstration,  including  the  validation  activities  conducted 
following  data  acquisitions,  provided  important  insights  regarding  the  appropriate  uses,  data 
processing  methods,  cost  and  performance  factors,  and  confidence  levels  for  both  technologies. 
All  positional  accuracy  specifications  were  met.  The  demonstration  provided  information  as  to 
the  advantages  and  limitations  of  these  technologies. 

Advantages  include  the  following: 

•  Rate  of  coverage  of  5,000  acres  or  greater  per  day 

•  Ability  to  delineate  MRS  and  munitions  and  explosives  of  concern  (MEC)-related 
features 

•  Contribution  to  planning  and  risk  assessment 

•  Increased  confidence  through  cross-validation  with  other  technologies 

•  Detailed  topographic  data  that  can  be  used  in  subsequent  phases  of  site 
investigation,  site  remediation,  and  range  management. 

Limitations  include  the  following: 

•  Inability  to  directly  detect  munitions  or  their  components. 

•  Detection  depends  on  the  persistent  or  continued  presence  of  surface  features. 
Features  can  be  subject  to  erosion  or  destruction  from  human  or  animal  activities. 

•  Orthophotos  do  not  contain  elevation  information. 

•  Orthophotos  do  not  “look  through”  vegetation,  and  LiDAR  point  densities  will  be 
lower  in  vegetated  areas. 

The  results  from  the  three  demonstration  sites  support  the  premise  of  the  WAA  Pilot  Program 
that  LiDAR  and  orthophotos  should  be  the  first  technologies  to  be  deployed  after  completion  of 
the  Archive  Search  Report  (ASR)  and  the  initial  CSM,  and  that  LiDAR  and  orthophoto 
acquisition  should  be  followed  with  technologies  that  directly  detect  munitions  components. 
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2.0  TECHNOLOGY  DESCRIPTION 

2.1  TECHNOLOGY  DEVELOPMENT  AND  APPLICATION 

2.1.1  Technology  Background 

LiDAR  is  a  well-established  airborne  technology  for  modeling  ground  surfaces.  Topographic 
LiDAR  was  first  developed  in  the  late  1960s  and  early  1970s  and  has  been  used  for  terrain 
profiling  since  the  mid-1980s.  LiDAR  has  been  in  wide  commercial  use  since  around  1993,  and 
the  accuracies  and  limitations  of  LiDAR  for  surface  modeling  are  well  documented. 

LiDAR  uses  the  time  of  return  for  a  laser  pulse  to  be  reflected  back  to  the  sensor  to  measure  the 
elevation  of  the  point  of  reflection.  Use  of  Global  Positioning  System  (GPS)  and  inertial 
measurement  unit  (IMU)  technology  to  locate  the  sensor  precisely  in  the  air  allows  for  the 
accurate  calculation  of  the  point  of  reflection  of  the  laser  signal  from  the  ground,  buildings,  or 
vegetation.  Multiple  returns  from  a  single  laser  pulse  can  be  detected,  increasing  the  chance  of 
sampling  the  ground  surface  through  gaps  in  vegetation.  Once  elevation  data  is  collected  in  the 
form  of  LiDAR  points,  surface  models  are  created  and  analyzed.  The  surface  modeling  process 
is  typically  conducted  using  standard  GIS  software  and  methods,  and  much  of  the  process  can  be 
successfully  automated.  LiDAR  vendors  typically  guarantee  a  vertical  accuracy  of  0. 1 5  m  and  a 
horizontal  accuracy  of  0.3-0.75  m. 

The  development  of  higher  speed  (50-100-kHz)  laser  scanners,  beginning  around  2002,  has 
significantly  improved  the  ability  of  LiDAR  to  locate  small  features.  Currently,  high-speed 
LiDAR  systems  are  being  used  to  characterize  objects  in  the  sub-meter  range,  such  as  power  line 
insulators  (see  Figure  1).  The  accuracy  and  data  density  of  current  LiDAR  systems  suggest  that 
the  technology  could  be  used  to  detect  ground  features  indicative  of  munitions  use,  including 
targets  and  craters,  and  that  the  presence  of  these  features  could  in  turn  be  used  to  develop  more 
accurate  locations  of  MEC. 
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Figure  1.  LiDAR  System  Operations. 


Digital  orthophotography  has  been  commercially  available  since  the  early  1980s,  with  steady 
improvement  in  the  resolution  (i.e.,  pixel  size)  and  precision  (i.e.,  pixel  placement)  of  the  images 
as  the  technology  of  digital  cameras,  GPS,  and  IMU  systems  has  advanced.  Since  the  mid- 
1990s,  image  size  has  advanced  from  1,500  pixels  across  an  image  to  4,500  pixels.  This  has 
allowed  for  increased  flying  heights  and  a  reduced  number  of  images  for  a  given  area,  with 
consequent  cost  savings.  Commensurate  with  this  improvement  has  been  a  twofold  increase  in 
the  accuracy  of  the  IMU,  allowing  for  accurate  positioning  of  image  pixels  at  a  higher  flying 
height  (see  Figure  2). 


Figure  2.  Helicopter-Mounted  LiDAR  and  Orthophoto  Sensor  Equipment. 
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Airborne  digital  cameras  have  been  successfully  integrated  with  LiDAR  sensors.  Cameras  with 
an  image  density  of  roughly  4,000  x  4,000  pixels  are  generally  favored  because  the  width  of  the 
images  collected  is  very  similar  to  that  of  the  typical  LiDAR  point  swath.  Once  collected, 
individual  digital  images  are  mosaiced  and  color-balanced,  and  the  resulting  composite  image  is 
orthorectified  using  the  LiDAR  data.  Orthorectification  allows  for  the  accurate  location  of  each 
photo  pixel,  eliminating  distortion  caused  by  camera  angle  and  topography.  Vendors  generally 
guarantee  a  horizontal  accuracy  of  3  pixel  widths  compared  to  ground  control  for 
orthophotography. 

Digital  images  are  collected  concurrently  with  LiDAR  and,  because  the  two  sensors  use  the  same 
GPS  and  IMU,  the  two  data  sets  can  be  integrated  very  accurately.  Vendors  generally  guarantee 
spatial  integration  of  orthophotos  and  LiDAR  within  2  pixel  widths.  Final  orthophoto  pixel  size 
depends  on  the  flight  altitude  and  the  camera  specifications;  helicopter-based  cameras  flying  at 
altitudes  of  400-450  m  are  capable  of  pixel  sizes  of  approximately  10  cm.  Smaller  pixel  sizes 
than  this  are  generally  impractical  due  to  the  low  flight  elevations  and  slow  flight  speeds 
required  to  collect  properly  overlapping  images,  and  the  very  large  numbers  of  images  that 
would  need  to  be  mosaiced. 

The  ability  to  produce  spatially  accurate  orthophotos  with  relatively  small  pixel  sizes  suggests 
that  this  technology  could  be  used  to  identify  munitions-related  features,  and  to  cross-validate 
technologies  such  as  LiDAR. 

2.2  PROCESS  DESCRIPTION 

2.2.1  Mobilization,  Installation,  and  Operational  Requirements 

At  all  three  sites,  flight  line  planning  for  the  LiDAR/orthophoto  and  LiDAR  only  flights  was 
conducted  by  Terra  Remote  Sensing,  Inc.  (TRSI),  the  ortho/LiDAR  vendor  used  for  this 
demonstration,  in  the  weeks  prior  to  mobilization.  Flight  lines  were  planned  to  ensure  complete 
site  coverage,  minimize  the  number  of  turns,  and  achieve  planned  overlap.  At  the  Kirtland  site, 
flight  lines  were  planned  to  minimize  interactions  between  data  collection  and  air  traffic  at  the 
Double  Eagle  Airport.  Digital  imagery  was  planned  for  acquisition  at  periods  of  low  sun  angle. 
Previous  testing  had  shown  that  the  shadows  created  by  low  sun  angle  were  useful  in  detecting 
shallow  features. 

The  LiDAR  and  orthophoto  sensor  system  was  installed  into  a  Bell  206B  helicopter  owned  by  a 
local  helicopter  vendor  (see  Figure  3).  Renting  helicopters  (and  pilots)  using  local  vendors  is  a 
standard  industry  practice  that  allows  the  LiDAR  vendor  to  ship  only  the  sensor  package  rather 
than  the  aircraft.  The  use  of  local  helicopter  vendors  also  allows  for  the  use  of  local  pilots  who 
have  better  knowledge  of  local  weather  patterns  and  flight  clearance  requirements. 
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Equipment  installation  Sensor  pod  mounted  below  helicopter,  control 

console  visible  through  window 


Figure  3.  LiDAR  and  Orthophoto  System  Installed  on  Helicopter. 

Data  collection  flights  were  begun  once  mobilization,  sensor  installation,  and  calibration  flights 
had  been  completed.  The  period  for  data  collection  included  an  additional  day  to  permit  re¬ 
acquisition  of  any  missed  or  erroneous  areas  discovered  during  daily  quality  assurance/quality 
control  (QA/QC)  review. 

2.2.2  Data  Processing  Steps 

Processing  of  the  sensor  output  to  create  LiDAR  points  was  performed  by  TRSI.  Following 
return  of  the  data  to  the  office,  calibration  factors  determined  in  the  field  were  checked,  fine- 
tuned,  and  applied  to  laser  range,  GPS,  and  IMU  data  to  produce  xyz  values  for  each  point. 
LiDAR  points  were  then  transformed  into  the  delivery  datum  and  projection,  and  coded  to 
indicate  returns  from  ground  versus  nonground  surfaces.  Additional  fields  for  each  point 
included  the  intensity  value,  GPS  date,  and  flight  line  number.  LiDAR  points  were  exported  as 
text  files  for  delivery  to  URS.  LiDAR  data  from  each  LiDAR  flight  was  processed  and  delivered 
separately  to  allow  for  separate  analysis  of  data  from  each  flight  altitude. 

Digital  image  processing  was  done  by  TRSI.  The  procedure  included  mosaicing  of  the 
individual  digital  images  collected  during  flight,  transformation  of  the  consolidated  image  to  the 
delivery  datum  and  projection,  orthorectification  using  the  LiDAR  data,  color  balancing,  and 
trimming  to  the  delivery  tiles. 

2.2.3  Selection  of  Analytical/Testing  Methods 

Analysis  of  LiDAR  data  is  performed  by  conversion  of  the  processed  LiDAR  points  to  usable 
GIS  products  such  as  surface  models,  contour  lines,  and  hillshades.  Creation  of  these  products 
and  their  analysis  were  accomplished  by  URS  using  ESRI’s  ArcGIS  software  suite.  ArcGIS  was 
the  only  standard  computer-aided  design  (CAD)  or  GIS  product  reviewed  that  would 
successfully  handle  the  large  number  of  LiDAR  points  collected.  Additionally,  ArcGIS  is  the 
GIS  package  most  widely  used  by  U.S.  government  agencies  and  private  contractors.  As  such,  it 
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is  appropriate  to  develop  analysis  methods  and  resulting  products  that  can  be  duplicated  by 
typical  federal  facilities  managers  using  existing  software  tools. 

Orthophoto  data  was  analyzed  by  visual  examination  of  the  image  to  locate  potential  MRS  and 
munitions-related  features. 

2.2.4  Safety  Issues 

Because  LiDAR  and  orthophotos  are  airborne  technologies,  they  avoid  the  typical  safety 
concerns  related  to  ground-based  munitions  investigation.  No  special  safety  issues  were 
encountered  during  data  collection  related  to  munitions  presence.  At  the  Kirtland  site,  flight 
operations  were  impacted  by  air  traffic  at  the  Double  Eagle  Airport,  located  between  the  north 
and  south  portions  of  the  study  area.  In  response,  TRSI  changed  its  planned  flight  lines  to  avoid 
the  airport  runway,  and  used  an  additional  spotter  in  the  helicopter  during  data  collection  flights. 

2.3  PREVIOUS  TESTING  OF  THE  TECHNOLOGY 

URS  and  TRSI  conducted  a  successful  demonstration  of  LiDAR  and  high-resolution  digital 
imagery  at  an  operational  U.S.  Navy  range  near  Boardman,  Oregon,  during  November  2004. 
Both  the  LiDAR  data  and  the  orthophotos  were  successful  in  detecting  patterns  of  surface 
disturbance  indicative  of  unexploded  ordnance  (UXO)/MEC  activities. 

Even  though  the  LiDAR  data  was  collected  at  relatively  low  density,  LiDAR  revealed 
depressions  such  as  craters  well.  LiDAR  was  very  successful  at  locating  target  features  such  as 
bull’s-eye  rings.  Disposal  craters  could  be  distinguished  from  bombing  practice  craters  by  their 
patterns  on  the  ground.  Orthophotos  cross-validated  most,  though  not  all,  features  visible  in  the 
LiDAR  surfaces.  However,  surface  models  created  from  LiDAR  data  could  be  analyzed  in  ways 
that  orthophoto  data  could  not  be,  such  as  measuring  the  depth  of  craters. 

2.4  ADVANTAGES  AND  LIMITATIONS  OF  THE  TECHNOLOGY 
2.4.1  Advantages 

•  Rate  of  coverage.  In  an  operational  setting,  data  collection  rates  of  5,000  acres 
or  greater  per  day  can  be  expected  for  LiDAR  and  orthophotos.  This  compares 
favorably  to  maximum  collection  rates  of  around  500  acres  per  day  for  helicopter- 
based  magnetometry,  and  20  acres  per  day  for  towed-array  magnetometry. 

•  Ability  to  delineate  MRS  and  MEC-related  features.  LiDAR  and 
orthophotography  successfully  revealed  MRS  and  MEC-related  surface  features  at 
both  demonstration  sites,  even  many  years  after  their  last  use.  In  the  case  of  the 
Kirtland  and  Camp  Beale  sites,  these  features  were  usually  not  visible  to 
observers  on  the  ground. 

•  Enhanced  planning  and  risk  assessment.  Because  they  can  cover  entire  sites 
relatively  quickly  and  at  lower  cost,  these  technologies  can  be  used  to  locate  and 
prioritize  appropriate  areas  for  use  of  more  costly  ground-based  technologies. 
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•  Increased  confidence  through  cross-validation.  LiDAR  and  orthophotos  data 
can  contribute  to  increased  confidence  levels  as  their  results  are  combined  with 
those  of  subsequent  technologies.  This  is  particularly  true  in  the  determination  of 
un-contaminated  areas  of  the  site. 

•  Other  benefits.  Both  technologies  provide  highly  detailed  topographic  data  that 
can  be  integrated  into  a  facility’s  CAD  or  GIS  system  and  used  in  subsequent 
phases  of  site  investigation,  site  remediation,  and  range  management. 

2.4.2  Limitations 

•  Munitions  detection.  Neither  LiDAR  nor  orthophotography  can  directly  detect 
munitions  or  their  components  such  as  scrap.  Rather,  these  technologies  rely  on 
detection  of  ground  surface  features  that  may  indicate  past  munitions  use. 
Consequently,  LiDAR  and  orthophotos  can  focus,  and  perhaps  reduce,  the  need 
for  further  investigation  with  magnetometers  or  EMI  sensors  but  cannot  eliminate 
it  altogether. 

•  Elevation  data.  Orthophotos  do  not  contain  elevation  information.  In  practice,  it 
is  sometimes  difficult  to  distinguish  small  surface  depressions  from  small  mounds 
or  shadows  using  orthophotos  alone. 

•  Vegetation  effects.  Since  both  LiDAR  and  orthophotos  are  light-based 
technologies,  neither  will  penetrate  vegetation.  Orthophotos  do  not  “look 
through”  vegetation,  and  LiDAR  point  densities  will  be  lower  in  vegetated  areas. 
However,  LiDAR  is  frequently  successful  in  penetrating  small  openings  between 
and  within  vegetation,  and  this  success  has  increased  with  the  speed  of  LiDAR 
sensors  and  the  development  of  the  ability  to  measure  multiple  returns. 
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3.0  DEMONSTRATION  DESIGN 


3.1  PERFORMANCE  OBJECTIVES 

The  primary  performance  objectives  for  these  technologies  were  to: 

•  Clarify  whether  and  to  what  extent  LiDAR  and  orthophotos  can  delineate  MRS 
boundaries  and  MEC-related  features,  and  contribute  to  focusing  and  prioritizing 
subsequent  low-altitude  and  ground-based  work 

•  Reveal  relationships  between  the  density  of  LiDAR  and  orthophoto  data,  their 
levels  of  cost,  and  their  ability  to  accurately  locate  MRS  boundaries  and  MEC- 
related  ground  features 

•  Clarify  whether  and  to  what  extent  LiDAR  and  orthophotos  can  verify,  reveal 
errors  in,  or  improve  the  accuracy  of  the  initial  CSM 

•  Contribute  data  and  analysis  to  the  overall  combination  of  technologies  used  in 
the  WAA  Pilot  Program,  in  a  manner  that  is  timely  to  the  application  of  the  other 
technologies  demonstrated,  in  formats  useable  by  other  demonstrators,  and  with 
sufficient  positional  accuracy  compared  to  project  control  points  to  allow 
meaningful  coordination  and  comparison. 

Specific  performance  criteria  and  performance  metrics  related  to  each  of  these  objectives  are 
documented  in  the  Final  Report  for  each  site. 

3.2  SELECTION  OF  TEST  SITES 

All  three  demonstration  sites  were  chosen  by  the  ESTCP  Program  Office.  Details  of  the  site 
selection  process  can  be  found  in  the  Final  Report  for  the  WAA  Pilot  Program. 

3.3  TEST  SITE  HISTORY,  CHARACTERISTICS,  AND  PRESENT  OPERATIONS 

The  first  demonstration  site  was  located  at  the  Kirtland  Air  Force  Base  PBR  located 
approximately  10  miles  west  of  Albuquerque,  New  Mexico.  Figure  4  shows  the  layout  of  the 
Kirtland  PBR  WAA  site.  The  site  is  part  of  a  much  larger  set  of  bombing  ranges  used  for 
training  purposes  during  World  War  II.  The  study  site  consisted  of  approximately  5,120  acres 
within  the  PBR,  located  in  two  parcels  to  the  north  and  south  of  the  Double  Eagle  Airport,  the 
primary  small  aircraft  airport  for  the  Albuquerque  area.  The  study  area  itself  is  currently 
undeveloped,  although  portions  are  planned  for  commercial  or  industrial  development,  and 
airport  expansion  into  the  study  is  possible. 
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Figure  4.  Kirtland  Precision  Bombing  Range  Site. 


The  target  areas  identified  in  the  CSM  for  the  Kirtland  site  included: 

•  Target  N-2,  a  1,000-ft-diameter  bull’s-eye  target  used  for  100-lb  practice  bombs 

•  Target  N-3,  a  1,000-ft-diameter  bull’s-eye  target  used  for  100-lb  practice  bombs 
and  for  scrap  storage 

•  The  New  Demolitions  Impact  Area  (NDIA),  a  1,000-ft-diameter  high  explosive 
(HE)  bull’s-eye  target 

•  The  Simulated  Oil  Refinery  Target  (SORT),  a  target  consisting  of  350-ft  x  350-ft 
rectangular  cells. 

The  second  demonstration  site  was  located  at  the  Former  Victorville  Army  Air  Force  Demolition 
Bombing  Target  (DBT)  Y  and  PBR  15  (Figure  5).  The  site  is  located  in  San  Bernardino  County, 
California,  approximately  42  miles  southeast  of  the  town  of  Victorville,  California.  This  site  lies 
within  a  much  larger  complex  of  approximately  23  targets  used  between  1942  and  1945.  The 
site  is  managed  by  the  U.S.  Bureau  of  Fand  Management  and  is  used  primarily  as  a  recreation 
area  for  off-road  vehicles,  camping,  and  target  shooting.  The  demonstration  site  encompasses 
approximately  5,640  acres. 
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Figure  5.  Victorville  DBT  Y  and  PBR  Target  15  Site. 


Two  target  areas  were  identified  in  the  initial  CSM  for  the  Victorville  site: 

•  Target  DBT  Y,  located  in  Means  Dry  Lake  bed  in  the  center  of  the  demonstration 
site,  identified  as  a  demolition  bomb  target  area  where  bombs  between  100  and 
2,000  lbs  were  used. 

•  Target  PBR  15,  a  suspected  bull’s-eye  target  located  in  the  southeast  portion  of 
the  demonstration  site  used  for  precision  bombing  practice.  According  to  the 
initial  CSM,  PBR  15  was  not  visited  during  the  ASR  site  visit  and  little  is  known 
about  the  area. 

The  Former  Camp  Beale  site  consists  of  87,672  acres  approximately  10  miles  east  of  Marysville, 
California,  between  Yuba  and  Nevada  counties  (see  Figure  6).  The  site  is  located  immediately  to 
the  east  of  Beale  Air  Force  Base.  The  demonstration  site  was  used  by  the  Department  of 
Defense  (DoD)  for  ground  ranges,  moving  target  ranges,  and  bombing  ranges  between  1943  and 
1959.  Historic  photographs  revealed  extensive  ground  disturbances,  expected  to  have  been 
created  during  previous  Munitions  Management  (MM)-related  activities.  Other  areas  from  the 
historical  photographs  were  noted  as  disturbed,  either  by  ground  scarring,  visible  craters,  or  other 
activities.  According  to  the  CSM,  cleanup  activities  were  conducted  in  1947  and  1958-1959. 
The  Former  Camp  Beale  site  was  excessed  and  sold  between  1959  and  1964  and  now  contains 
both  private  land  and  state  land  located  within  the  Spenceville  Wildlife  Reserve. 
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Figure  6.  Former  Camp  Beale  Demonstration  Site  Location. 

The  CSM  identified  numerous  bombing  ranges,  firing  ranges,  and  training  ranges,  often 
overlapping,  but  provided  little  to  no  information  about  specific  target  locations. 

3.4  ANALYTICAL  PROCEDURES 

At  all  three  sites,  analysis  of  the  LiDAR  and  orthophoto  data  include  the  following  steps: 

•  Review  of  the  LiDAR  surface  models  and  orthophotos  to  detect  missing  data, 
spatial  discrepancies,  noise  effects,  or  other  quality  problems.  LiDAR  and 
orthophoto  spatial  accuracy  reported  by  the  vendor  was  independently  checked 
using  the  site  survey  control  points. 

•  Visual  examination  of  each  LiDAR  and  orthophoto  data  set  for  potential  MRS. 

•  Visual  examination  of  each  LiDAR  and  orthophoto  data  set  for  individual  features 

such  as  craters. 

•  Suggested  modifications  to  the  MRS  or  target  area  boundaries  from  the  CSM. 

•  Examination  of  the  effects  of  data  density  and  other  performance  parameters, 

using  the  control  points,  calibration  craters,  and  vertical  control  structures. 

To  reduce  potential  bias,  the  initial  data  analysis  was  conducted  by  GIS  technicians  without 
reference  to  the  CSM.  Results  of  the  initial  analysis  were  reviewed  by  staff  with  expertise  in 
UXO. 

Detailed  description  of  the  analytical  procedures  used  is  found  in  the  Final  Report  for  the 
Kirtland  and  Victorville  sites,  and  the  Final  Report  Addendum  for  the  Former  Camp  Beale  site. 
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4.0  PERFORMANCE  ASSESSMENT 

4.1  PERFORMANCE  DATA 

4.1.1  MRS  and  Feature  Detection,  Kirtland  Site 

At  the  Kirtland  site,  data  collection  flights  took  place  on  August  9,  10,  and  11,  2005,  with  a  total 
of  177  flight  lines  collected.  In  order  to  test  data  density  effects,  flight  lines  were  collected  at 
three  altitudes:  900  m,  450  m,  and  300  m.  In  order  to  test  the  effect  of  flight  line  orientation,  two 
sets  of  flight  lines  were  collected  at  300  m.  For  the  north  portion  of  the  project  area,  the  two 
flight  line  sets  were  flown  perpendicular  to  each  other;  for  the  south  portion  they  were  flown 
parallel.  Digital  images  were  collected  concurrently  with  LiDAR  during  the  900  m  and  450  m 
flights,  in  order  to  produce  orthophotos  with  10-cm  (4-in)  and  20-cm  (8-in)  pixel  sizes. 

The  Kirtland  CSM  described  four  historic  bombing  sites,  one  of  which  did  not  have  a  well- 
defined  location.  All  four  were  visible  in  the  LiDAR  surface  models,  and  one  was  visible  in  the 
orthophotos.  Additionally,  LiDAR  data  showed  the  presence  of  subtargets  not  mentioned  in  the 
CSM  and  a  potential  additional  target  area.  Target  objects  were  detected  despite  the  fact  that  the 
berms  making  up  the  target  were  from  15-20  cm  in  height  and  generally  not  visible  to  crews  on 
the  ground.  In  addition  to  the  main  target  features,  1 5  additional  sites  of  interest  were  detected 
using  the  LiDAR  data,  including  seven  in  the  north  portion  of  the  site  and  eight  in  the  south 
portion.  Most  consisted  of  isolated  groups  of  potential  craters. 

The  Kirtland  site  contained  just  over  100  small  features  that  were  potentially  related  to  munitions 
use,  mostly  isolated  potential  craters.  The  site  also  contained  numerous  linear  features,  mostly 
jeep  trails  and  other  vehicle  tracks.  Potential  features  were  visually  identified  from  the  LiDAR 
hillshades  and  the  orthophotos. 

A  sample  of  the  target  objects  detected  is  shown  in  Figures  7-10. 
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Figure  7.  Kirtland  Site  Bombing  Targets. 
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4.1.2  MRS  and  Feature  Detection,  Victorville  Site 

At  the  Victorville  site,  orthophoto  data  collection  flights  took  place  on  January  24  and  25,  2006, 
and  LiDAR  data  collection  flights  took  place  on  February  3  and  4,  2006.  A  total  of  45  flight 
lines  was  collected  at  two  altitudes:  450  m  and  300  m.  The  two  flights  were  flown  perpendicular 
to  each  other.  Digital  images  were  collected  concurrently  with  the  450-m  flight  in  order  to 
produce  orthophotos  with  10-cm  pixels. 

The  Victorville  CSM  described  two  historic  bombing  sites,  a  demolition  bombing  area  located  in 
a  dry  lake  bed,  and  a  practice  bombing  target.  Both  targets  were  clearly  visible  using  LiDAR  and 
orthophoto  data.  No  additional  sites  of  interest  were  detected  at  Victorville  using  these 
technologies.  LiDAR  data  showed  numerous  craters  in  and  near  the  DBT.  These  were  used  to 
refine  the  boundaries  of  the  MRS  for  the  target. 


Target  PBR  15,  orthophoto. 


Target  PBR  15,  LiDAR  intensity  image. 


Target  DBT  Y.  The  blue  line  shows  the  original 
target  boundary  from  the  CSM.  Red  dots  show  the 
crater  locations  from  the  LiDAR  data,  and  the  red 
line  shows  the  revised  MRS  boundary. 


Target  DBT  Y,  individual  craters 


Figure  8.  Victorvill 


e  Site  Target  Areas. 
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4.1.3  MRS  and  Feature  Detection,  Former  Camp  Beale  Site 

At  the  Former  Camp  Beale  site,  data  collection  flights  took  place  on  July  22  through  26,  2006, 
with  a  total  of  120  flight  lines  collected  at  altitudes  of  300  and  450  m.  Orthophoto  data  was 
collected  concurrently  with  LiDAR  during  the  450-m  flight  in  order  to  produce  images  with  a 
10-cm  pixel  size. 

The  Former  Camp  Beale  CSM  described  a  wide  variety  of  bombing  areas,  training  ranges,  and 
firing  ranges,  often  overlapping.  No  specific  target  locations  were  given.  Three  bombing  areas, 
including  one  bull’s-eye  target,  were  located  using  the  LiDAR  and  orthophoto  data,  along  with 
features  from  two  firing  ranges.  The  bull’s-eye  target  was  located  outside  the  mapped  bombing 
practice  areas  in  the  CSM.  Approximately  1,000  individual  features  were  identified  (not 
including  craters  in  two  crater  fields).  Many  individual  features  were  ambiguous  and  their  origin 
could  not  be  determined  without  further  investigation.  A  sample  of  the  target  areas  detected  is 
given  in  the  following  figures: 


Potential  Firing  Range  2.  This  area  consists  of  13  pairs  of  fan-shaped  level  areas,  along  with  seven  bunkers. 
The  area  is  U.S.  Air  Force  used  for  small  arms,  signals,  booby  traps,  trip  flares,  and  other  pyrotechnics. 


Bombing  Area  1.  This  is  a  bull’s  eye  target  consisting  of  four  rings,  the  largest  of  which  is  approximately 
300  m  (984  ft)  in  diameter.  The  area  contains  numerous  craters  ranging  up  to  approximately  3  m  in  diameter 
_ and  0.1  -m  deep. _ 
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Figure  9.  Former  Camp  Beale  Target  Areas. 
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Bombing  Area  1  is  located  between  Target  3  and  Navy  Target  T-63 
in  the  CSM,  which  was  used  for  HE  bombing  practice  from  1948  to 
1955.  It  appears  most  likely  that  this  is  the  HE  bombing  target,  and 
the  maps  accompanying  the  initial  CSM  are  somewhat  erroneous. 


Figure  10.  Bombing  Area  1  with  CSM  Areas. 


4.1.4  Performance  Factors:  Data  Density  Effects 

LiDAR  and  orthophoto  data  density  were  found  to  have  a  significant  effect  on  the  performance 
of  both  technologies,  up  to  a  point  of  diminishing  returns.  For  orthophotos,  images  with  a  10-cm 
pixel  size  performed  significantly  better  than  the  20-cm  images  collected  at  the  Victorville  site, 
or  the  30-cm  images  that  were  previously  available  for  the  Former  Camp  Beale  site  (see 
Figure  11). 
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Kirtland  site,  10-cm  orthophoto  with  calibration  crater 
locations.  Calibration  craters  are  1.5  m,  1.0  m,  and  0.3  m  in 
diameters. 


t 

I* 
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Kirtland  site,  20-cm  orthophoto  with  calibration  crater 
locations. 


Former  Camp  Beale  site,  10-cm  pixel  orthophoto  with  bull’s 
eye  aiming  target. 


Former  Camp  Beale  site,  30-cm  (1  -ft)  pixel  orthophoto  with 
bull’s  eye  aiming  target. 


Figure  11.  Orthophoto  Data  Density  Results. 

For  LiDAR,  increasing  the  LiDAR  point  density  had  only  a  slight  impact  on  detecting  large 
target  items  but  improved  the  ability  to  detect  smaller  items  such  as  the  calibration  craters,  as 
shown  in  Figure  12.  However,  LiDAR  was  not  able  to  reliably  detect  the  0.3-m  ( 1  -ft)  calibration 
craters  at  any  of  the  data  densities  collected.  This  was  true  even  at  the  extremely  high  data 
density  of  13.8  LiDAR  points/m2  collected  at  the  Former  Camp  Beale  site,  a  value  well  above 
that  necessary  to  delineate  craters. 
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Kirtland  site,  900  m  (1.6  pts/m2) 
LiDAR  hillshade. 


Kirtland  site,  450  m  (4.1  pts/m2) 
LiDAR  hillshade. 


Kirtland  site,  300  m  (6.1  pts/m2) 
LiDAR  hillshade. 


0.3  m 
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0.3  m 

0.3  m 

1.5  m 


1.0  m 
1.0  m 

Former  Camp  Beale  site,  13.8  pts/m2  LiDAR  hillshade  with 
calibration  craters. 


Portion  of  the  image  to  the  left  showing  LiDAR  points,  a 
1.5-m  calibration  crater  and  two  0.3-m  calibration  craters. 
Even  at  this  very  high  data  density,  one  of  the  two 
calibration  craters  receives  no  LiDAR  points. 


Figure  12.  LiDAR  Data  Density  Results. 


LiDAR  data  density  was  found  to  vary  considerably  across  the  test  area.  This  finding  may  have 
some  implications  for  detecting  small  individual  features.  LiDAR  density  variations  and  effects 
are  discussed  in  detail  in  the  Final  Report  for  the  Kirtland  and  Victorville  sites,  and  the  Final 
Report  Addendum  for  the  Former  Camp  Beale  site. 

4.1.5  Performance  Factors:  Flight  Line  Orientation 

Flight  line  orientation,  or  the  direction  the  aircraft  was  flying  while  the  data  was  being  gathered, 
had  an  impact  on  the  detection  of  faint  linear  features  such  as  dirt  roads  and  shallow  berms. 
Flight  line  orientation  had  no  discemable  impact  on  crater  detection  (see  Figure  13). 
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Kirtland  site,  east-west  LiDAR  flight  lines.  East- 
west  roads  appear  more  clearly. 


Kirtland  site,  north-south  LiDAR  flight  lines.  East-west 
roads  appear  less  clearly. 


Figure  13.  Flight  Line  Effects. 


4.1.6  Performance  Factors:  Vegetation 

As  a  light-based  technology,  LiDAR  does  not  penetrate  vegetation.  Nevertheless,  LiDAR  is 
often  successfully  used  to  model  the  ground  surface  under  vegetation  since  LiDAR  points  will 
often  fall  in  the  many  gaps  between  foliage.  In  practice,  LiDAR  has  been  used  to  model  ground 
surfaces  in  all  but  truly  closed  canopies. 

However,  while  some  LiDAR  points  will  penetrate  to  the  ground  surface  in  vegetated  areas,  most 
will  not.  Consequently,  the  surface  model  under  vegetative  cover  will  be  less  detailed.  As  a 
preliminary  attempt  to  quantify  this  effect,  two  areas  of  %  km2  were  modeled.  A  grid  of  2  m 
cells  was  created,  with  each  cell  assigned  the  percentage  of  LiDAR  returns  that  were  reflected 
from  a  point  3  ft  or  higher  from  the  ground  surface.  At  these  two  areas,  the  forested  areas 
blocked  50%  -  80%  of  the  LiDAR  points,  while  in  dense  brush  over  90%  of  the  LiDAR  points 
were  blocked.  Under  the  trees,  good  modeling  of  the  ground  surface  was  nevertheless  achieved. 
This  may  have  been  a  result  of  the  high  overall  LiDAR  data  densities  involved.  In  the  area  of 
dense  brush,  the  vegetation  effect  caused  serious  degradation  of  the  surface  model.  The  effect  is 
shown  in  the  images  in  Figure  14. 
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Orthophoto  showing  open  area,  light  tree  cover,  and  brush. 


LiDAR  ground  surface  model.  The  surface  model  for  the 
brushy  area  is  severely  degraded. 


All  points  LiDAR  surface  model. 


Density  map  showing  the  percentage  of  LiDAR  points 
reflected  from  vegetation  over  3  ft  in  height. 


Figure  14.  Vegetation  Effects  on  LiDAR  Surface  Models. 

4.1.7  Performance  Factors:  Data  Artifacts  and  Noise  Effects 

At  all  three  sites,  the  LiDAR  data  showed  “corduroy”  effects  roughly  0.05-m  deep  in  areas  of 
relatively  flat,  smooth  terrain  (see  Figure  15).  This  is  a  common  LiDAR  artifact,  generally 
believed  to  result  from  small  errors  in  the  GPS,  IMU,  and  laser  range  finder  that  cannot  be 
adjusted  out  during  data  processing.  The  size  of  the  anomaly  is  well  within  the  vertical  accuracy 
specifications  for  LiDAR  data.  The  images  below  show  the  relationship  between  the  observed 
“corduroy  stripes”  in  the  modeled  ground  surface  and  the  lines  of  LiDAR  points. 
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Former  Camp  Beale  site,  “corduroy”  effect  Former  Camp  Beale  site,  “corduroy”  effect  with  LiDAR 

points 


Figure  15.  LiDAR  Data  Artifacts. 

4.2  PERFORMANCE  CRITERIA 

LiDAR  and  orthophoto  data  collected  for  this  demonstration  met  the  performance  criteria  related 
to  data  collection,  data  procession,  site  coverage  and  positional  accuracy  established  for  each 
site.  Detailed  results  are  given  in  the  Final  Report  for  the  Kirtland  and  Victorville  sites  and  the 
Final  Report  Addendum  for  the  Former  Camp  Beale  site. 

4.3  DATA  ASSESSMENT 

The  performance  data  from  the  three  demonstration  sites  largely  supported  the  performance 
claims  for  LiDAR  and  orthophotos  at  munitions  sites. 

•  Performance.  The  LiDAR  and  orthophoto  data  met  the  performance  objectives  in 
Table  3-1  of  the  Demonstration  Plan  for  each  site. 

The  two  technologies  successfully  detected  and  refined  the  boundaries  of 
MRS  at  each  site.  LiDAR  was  more  useful  in  MRS  detection  and 
delineation  than  orthophotos;  however,  orthophotos  added  detection 
capabilities  in  some  cases  that  LiDAR  did  not. 

LiDAR  was  successfully  used  in  the  detection  of  individual  features,  with 
the  success  of  detection  increasing  as  a  function  of  LiDAR  data  density  to 
a  point  of  diminishing  returns.  Detection  ability  increased  substantially 
between  1.5  pts/m2  and  4.5  pts/m2,  and  slightly  between  4.5  pts/m2  and  6.0 
pts/m2.  Detection  ability  did  not  increase  past  this  point.  Although  the 
higher  densities  collected  at  the  Former  Camp  Beale  site  (13.8  pts/m2) 
resulted  in  very  detailed  surface  models,  the  added  data  density  did  not 
result  in  additional  feature  detection. 

•  Personnel/training  requirements.  Personnel  and  training  requirements  did  not 
differ  from  standard  industry  claims.  The  vendor  provided  qualified  personnel 
and  no  issues  were  noted. 
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•  Health  and  safety  requirements.  Project  health  and  safety  requirements  did  not 
differ  from  standard  industry  claims.  As  airborne  technologies,  LiDAR  and 
orthophotos  did  not  pose  the  same  safety  concerns  as  ground-based  technologies. 
The  only  health  and  safety  issue  encountered  was  at  the  Kirtland  Air  Force  Base 
PBR  site,  where  air  traffic  from  the  nearby  Double  Eagle  Airport  made  it 
necessary  to  modify  the  originally  planned  flight  lines. 

•  Ease  of  operation.  Ease  of  operation  conformed  to  industry  claims  for  both 
technologies.  Mobilization  and  demobilization  occurred  without  incident,  and 
data  was  collected,  processed,  and  delivered  in  conformance  with  established 
specifications  and  within  the  established  schedule. 

•  Limitations.  The  limitations  of  LiDAR  and  orthophotos  did  not  differ  from  the 
general  principles  presented  in  Section  2.4,  Advantages  and  Limitations  of  the 
Technology.  The  demonstration  further  clarified  these  limitations,  as  presented  in 
Section  4.1,  Performance  Data.  Specifically: 

LiDAR  and  orthophotos  identified  individual  features  to  approximately 
1.0  m  in  size,  but  not  smaller.  This  was  true  even  at  the  very  high  data 
density  collected  at  the  Former  Camp  Beale  site. 

Vegetation  at  the  Former  Camp  Beale  site  partially  obscured  the  ground 
surface  in  some  areas.  Vegetation  cover  limited  the  usefulness  of 
orthophotos  in  these  areas  since  the  images  primarily  showed  vegetation 
rather  than  the  ground.  LiDAR-based  surface  models  were  very  good 
quality  under  the  relatively  light  tree  cover  and  more  problematic  under 
brush  along  stream  channels. 

LiDAR-based  surface  models  showed  small  noise  effects,  well  within  the 
positional  accuracy  specifications  of  the  technology. 

4.4  TECHNOLOGY  COMPARISON 

The  WAA  Pilot  Program  provided  an  opportunity  to  compare  the  performance  characteristics  of 
several  innovative  and  existing  approaches  to  site  assessment,  including  helicopter 
magnetometry,  towed-array  magnetometry,  and  electromagnetic  (EM)  sensing,  intrusive 
investigation,  and  field  reconnaissance.  Each  of  these  approaches  provided  useful  data  that  the 
others  did  not,  and  the  combination  provided  a  higher  degree  of  certainty  than  any  single 
approach. 

The  primary  comparative  results  were  that  at  the  Kirtland  site,  helicopter  magnetometry  showed 
two  areas  of  concentrated  magnetic  anomalies  within  one  of  the  target  areas  that  was  not 
detected  by  LiDAR  or  orthophotos.  However,  at  both  the  Kirtland  and  Victorville  sites,  LiDAR 
showed  areas  of  extensive  cratering  that  had  only  a  low  density  of  anomalies  in  the  helicopter 
magnetometry.  Ground-based  transects  cross-validated  the  helicopter  magnetometry  results  and 
established  background  levels  of  magnetic  anomalies  that  were  beyond  the  resolution  of  the 
helicopter  magnetometry  (see  Figure  16). 
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Helicopter  and  transect  results  for  Kirtland  Target  N-3.  Source:  ESTCP.  Two  areas  of  concentrated  magnetic  anomalies,  N3-AOP-7  and 
N3-AOI-8,  were  not  visible  in  the  LiDAR  and  orthophotos  data  since  these  areas  showed  no  surface  disturbance. 
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Helicopter  and  transect  results  for  Kirtland  Target  NDIA.  Source:  ESTCP.  This  area  shows  a  low  density  of  magnetic  anomalies  compared 
_ to  background,  although  the  target  cross-hairs  are  visible  in  the  orthophotos  and  craters  can  be  seen  in  the  LiDAR  data. _ 


*  AOI  =  area  of  interest 


Figure  16.  Example  Helicopter  and  Ground-Based  Transect  Results. 
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5.0  COST  ASSESSMENT 


5.1  COST  REPORTING 

Table  1  presents  actual  costs  for  the  Kirtland,  Victorville,  and  Former  Camp  Beale 
demonstration  sites,  and  estimated  costs  for  production  sites  of  several  additional  sizes.  The 
figures  for  two  production  sites  are  planning-level  estimates,  assessed  to  be  accurate  +/-  20%  at 
the  time  of  this  report.  Per-acre  costs  for  the  Kirtland  site  were  higher  since  four  rather  than  two 
LiDAR  flights  were  conducted,  and  one  rather  than  two  orthophoto  sets  were  created.  The 
Victorville  configuration,  with  one  LiDAR/orthophoto  flight  and  one  additional  LiDAR-only 
flight,  is  considered  representative  for  a  production  site  where  it  is  important  to  detect  both 
targets  and  individual  small  features  and  was  used  at  the  Camp  Beale  site.  Estimates  for  the 
larger  sites  are  based  on  URS’  previous  experience  with  LiDAR  and  orthophoto  collection 
projects  and  interviews  with  industry  sources.  All  figures  are  in  2006  U.S.  dollars  and  costs 
were  updated  in  June  2007.  All  projects  listed  can  be  completed  in  less  than  one  year;  therefore, 
no  discount  factor  has  been  applied  to  the  figures. 
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Table  1.  Actual  and  Projected  Costs, 


Project  Parameters 

1,000- 
Acre  Site 

Kirtland 

Victorville 

10,000- 
Acre  Site 

Former 

Camp 

Beale 

50,000- 
Acre  Site 

115,000- 
Acre  Site 

250,000- 
Acre  Site 

Project  area  size 
(acres) 

1,000 

5,000 

5,640 

10,000 

18,000 

50,000 

115,000 

250,000 

Project  area  size 
(hectares) 

404 

2,020 

2,279 

4,040 

7,272 

20,200 

46,460 

101,000 

LiDAR  flights: 

300  m  (LiDAR  only) 

2 

1 

1 

1 

1 

1 

1 

450  m  (LiDAR  and  10- 
cm  pixel  imagery) 

1 

1 

1 

1 

1 

1 

1 

1 

900  m  (LiDAR  and  20- 
cm  pixel  imagery) 

0 

1 

0 

0 

0 

0 

0 

0 

Total  LiDAR  flights 

2 

4 

2 

2 

2 

2 

2 

2 

Total  LiDAR  point 
density  (pts/m2) 

~10 

20 

~10 

~10 

14 

~10 

~10 

~10 

Orthophoto  pixel  size 
(cm) 

1 

0 

10  and  20 

10 

10 

10 

10 

10 

10 

Costs: 

Fixed  Costs 

$ 

Mob/demob 

15,000 

15,600 

23,100 

21,000 

21,800 

30,000 

45,000 

50,000 

Planning/preparation 

8,000 

15,000 

9,200 

12,000 

15,000 

15,000 

20,000 

22,000 

Project  management 

10,000 

15,000 

10,000 

10,000 

25,000 

40,000 

80,000 

100,000 

Site  work 

0 

0 

0 

0 

0 

0 

Equipment  cost 

0 

0 

0 

0 

0 

0 

Start-up  and  testing 

0 

0 

0 

0 

0 

0 

Subtotal  fixed  costs 

33,000 

42,300 

43,000 

61,800 

85,000 

145,000 

Variable  Costs 

$ 

Data  acquisition 

25,000 

34,100 

64,000 

85,300 

160,000 

355,000 

Data  processing 

18,000 

45,800 

35,200 

60,000 

102,900 

250,000 

575,000 

Data  analysis  and  GIS 
products 

10,000 

94,300 

30,000 

38,000 

68,100 

150,000 

220,000 

260,000 

Data  reporting  and 
documentation 

8,000 

13,600 

8,500 

9,000 

12,000 

15,000 

25,000 

30,000 

Materials  and 
consumables 

500 

1,500 

1,000 

1,500 

5,000 

10,000 

15,000 

Other  Direct  Costs 

$ 

0 

0 

0 

0 

0 

0 

Subtotal  variable 
costs 

$61,500 

195,100 

108,800 

171,000 

269,800 

580,000 

1,185,000 

1,705,000 

Total  project  cost 

$94,500 

240,700 

151,100 

214,000 

331,600 

665,000 

1,330,000 

1,877,000 

Total  per/acre  cost 

$94.50 

48.14 

26.79 

21.40 

18.42 

13.30 

11.57 

7.51 

Total  per/hectare  cost 

$233.91 

66.31 

52.97 

45.60 

32.92 

28.63 

18.58 

5.2  COST  ANALYSIS 
5.2.1  Cost  drivers 

The  major  cost  drivers  for  the  Camp  Beale  site  largely  confirmed  the  findings  from  the  Kirtland 
and  Victorville  sites.  These  were: 

•  LiDAR  data  density  required.  For  the  Kirtland  site,  four  LiDAR  flights  were 
conducted — two  concurrently  with  digital  imagery  collection  and  two  LiDAR- 
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only  flights.  For  the  Victorville  site,  one  LiDAR/orthophoto  flight  and  one 
LiDAR-only  flight  were  conducted.  For  Camp  Beale,  two  LiDAR  flights  were 
conducted. 

•  Orthophoto  data  density  required.  For  the  Kirtland  site,  two  sets  of  digital 
images  were  collected,  and  orthophotos  were  created  at  10-cm  and  20-cm  pixel 
sizes.  For  the  Victorville  site,  only  10-cm  pixel  size  was  collected.  For  the 
Victorville  and  Former  Camp  Beale  sites,  only  10-cm  pixel  orthophotos  were 
acquired. 

•  Accuracy  and  precision  requirements.  A  higher  level  of  survey  control  was 
needed  at  the  Camp  Beale  site  than  for  production  sites,  including  collecting  16 
control  points,  10  test  craters,  and  four  vertical  control  structures.  For  production 
projects,  fewer  survey  control  points  and  vertical  control  structures  would  likely 
be  needed.  However,  the  cost  of  project  control  is  small  relative  to  that  of  data 
acquisition  and  processing,  and  the  cost  savings  would  be  relatively  minor. 

•  Site  location  and  logistics.  The  Camp  Beale  site  location  affected  project  costs 
both  positively  and  negatively.  The  site  is  situated  close  enough  to  the  Yuba 
Airport  so  as  not  to  require  establishing  a  fuel  cache  on  site  or  to  require  the 
aircraft  to  land.  Negative  factors  included  the  proximity  of  the  Phased  Array 
Warning  System  (PAWS)  radar  installation,  which  interfered  with  the  LiDAR 
equipment  requiring  additional  data  processing,  and  the  excessive  ambient 
temperatures  experienced  during  the  data  acquisition  flights  which  limited  the 
flights  to  the  early  morning  hours  and  required  slower  aircraft  speeds. 

In  addition  to  the  cost  drivers  listed  above,  costs  for  production  sites  will  be  affected  by  the 
following  additional  factors: 

•  Site  size.  Larger  sites  achieve  cost  savings  through  amortization  of  fixed  costs 
such  as  mobilization  and  project  planning,  as  well  as  through  increased  efficiency 
in  data  acquisition  and  processing.  This  effect  can  be  seen  in  Table  1. 

•  Vegetation  conditions.  Highly  vegetated  sites  may  have  higher  costs  due  to  the 
requirement  for  additional  LiDAR  passes  to  achieve  sufficient  density  of  points 
reaching  the  ground  surface.  Alternatively,  it  may  be  possible  to  achieve 
sufficient  vegetation  penetration  by  specifying  the  use  of  higher  speed  sensor 
equipment. 

•  Permitting  and  site  access  constraints.  DoD  sites  with  sensitive,  high-security 
areas  may  have  higher  costs.  However,  such  conditions  would  typically  affect 
only  pre-flight  planning  and  equipment  mobilization  costs  rather  than  data 
acquisition,  processing,  and  analysis  costs.  Sites  with  environmental  constraints 
do  not  normally  impose  significantly  higher  costs  for  LiDAR  and 
orthophotography  since  the  airborne  nature  of  the  technologies  does  not  typically 
affect  sensitive  species  or  environments. 
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5.2.2  Cost  Sensitivities  and  Additional  Potential  Savings 

Additional  savings  could  be  realized  through  either  of  the  following  methods: 


•  Acquiring  orthophotography  with  a  larger  pixel  size.  The  cost  of  acquiring  and 
processing  orthophotography  rises  dramatically  for  smaller  pixel  sizes,  and 
acquiring  orthophotos  at  20-cm  pixel  size  rather  than  10-cm  would  reduce  the 
data  acquisition  and  processing  costs  by  30-35%.  The  utility  of  such  photos 
would  be  lower  since  their  resolution  will  not  allow  discrimination  of  smaller 
features.  At  highly  vegetated  sites,  orthophotos  are  inherently  less  useful,  and 
orthophotos  with  larger  pixel  sizes  may  be  acceptable  or  orthophoto  collection 
may  be  eliminated  altogether  if  pre-existing  orthophotography  is  available  and  its 
positional  accuracy  can  be  verified.  However,  at  relatively  open-sky  sites,  site 
managers  should  consider  acquiring  10-cm  pixel  orthophotos.  Experience  during 
both  phases  of  the  WAA  Pilot  Program  has  shown  that  these  are  significantly 
more  useful  than  orthophotos  with  larger  pixel  sizes. 

•  Acquiring  lower-density  LiDAR  data.  Eliminating  the  assumed  second  LiDAR 
flight  and  thus  collecting  LiDAR  only  with  the  10-cm  orthophoto  imagery  would 
reduce  costs  by  25  to  30%.  The  ability  of  the  resulting  LiDAR  data  set  to 
discriminate  features  would  be  reduced;  however,  this  might  be  appropriate  if  the 
LiDAR  data  was  to  be  used  only  to  discriminate  large  features  such  as  bombing 
targets  or  roads  rather  than  smaller  features  such  as  craters.  Alternatively,  a  faster 
LiDAR  sensor  could  be  used  that  could  meet  LiDAR  data  density  requirements 
from  a  single  pass. 

Some  additional  cost  savings  could  potentially  be  achieved  by  establishing  Service-  or  DoD- 
wide  standards  for  data  acquisition,  GIS  data  product  creation,  data  delivery  formats,  and  project 
reporting. 

5.3  COST  COMPARISON 

Cost  comparisons  with  the  other  innovative  technologies  demonstrated  as  part  of  the  ESTCP 
WAA  Pilot  Program  will  be  made  in  the  Final  Report  for  the  WAA  Pilot  Program. 
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6.0  IMPLEMENTATION  ISSUES 


6.1  COST  OBSERVATIONS 

Key  factors  that  affected  project  costs  are  summarized  in  Section  5.2,  Cost  Analysis,  including 
sensitivity  to  site-specific  conditions  and  areas  for  potentially  reducing  costs  in  future 
applications.  Learning  curve  effects  were  relatively  small  over  the  three  demonstration  sites  due 
to  the  use  of  experienced  vendors. 

6.2  PERFORMANCE  OBSERVATIONS 

LiDAR  and  orthophoto  data  collected  for  this  demonstration  met  the  performance  criteria  related 
to  data  collection,  data  procession,  site  coverage,  and  positional  accuracy  established  for  each 
site.  Details  are  presented  in  the  Final  Report  for  the  Kirtland  and  Victorville  site  and  the  Final 
Report  Addendum  for  the  Former  Camp  Beale  site. 

6.3  SCALE-UP 

No  technical  impediments  were  noted  that  would  affect  scale-up  from  demonstration-scale  to 
full-scale  implementation.  Both  LiDAR  and  orthophototpgraphy  are  in  wide  commercial  use  on 
large  sites  nationwide. 

6.4  OTHER  SIGNIFICANT  OBSERVATIONS 

The  Former  Camp  Beale  site  presented  a  much  larger  number  of  ambiguous  features  than  the 
first  two  sites  examined.  This  site  may  be  representative  of  some  formerly  used  defense  sites 
that  have  a  complex  history  of  munitions-  and  non-munitions  use.  Most  of  the  ambiguous 
features  identified  were  shallow  depressions  that  resembled  craters  in  the  LiDAR  and  orthophoto 
data,  which  could  not  be  distinguished  from  munitions  craters  using  LiDAR  and  orthophotos 
alone.  Subsequent  field  work  showed  that  only  a  minority  of  these  had  a  positive  magnetic 
response.  However,  LiDAR  and  orthophotos  were  successfully  used  to  locate  these  features  for 
subsequent  field  investigation.  Given  the  large  size  of  the  site,  the  cost  savings  from  identifying 
and  prioritizing  these  ambiguous  features  would  be  substantial  compared  to  using  other  methods. 

6.5  LESSONS  LEARNED 

Results  from  all  three  sites  support  the  general  premise  of  the  WAA  Pilot  Program  that  LiDAR 
and  orthophotos  should  be  the  first  technologies  to  be  deployed  after  completion  of  the  ASR  and 
the  initial  CSM.  At  all  three  sites,  LiDAR  and  orthophotos  were  successful  at  revealing  and 
verifying  the  broad  picture  of  munitions  use.  LiDAR,  especially,  was  very  successful  at 
delineating  targets  and  crater  fields,  along  with  ambiguous  features  that  warranted  investigation. 
The  two  technologies  complemented  each  other  well,  each  providing  data  that  the  other  did  not. 
Both  technologies  provided  base  data  that  were  useful  for  planning  subsequent  investigation. 
Since  vendors  generally  offer  the  two  technologies  together,  it  makes  sense  to  acquire  both  at 
future  production  sites. 

At  the  Kirtland  site,  two  AOIs  were  identified  using  magnetometry  that  were  not  detected  using 
LiDAR  and  orthophotos,  presumably  because  these  areas  did  not  leave  any  indications  on  the 
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ground  surface.  However,  at  the  Kirtland  and  Victorville  sites,  LiDAR  and  orthophotos  were 
used  to  identify  bombing  targets  that  had  little  to  no  magnetic  signature,  possibly  due  to  earlier 
cleanup  efforts.  In  this  sense,  LiDAR  and  orthophotos  complemented  the  magnetometer  and 
EMI  technologies,  and  the  combination  resulted  in  a  higher  level  of  confidence  than  either 
separately. 

The  Former  Camp  Beale  site  was  a  logical  extension  of  the  WAA  Pilot  Program  to  a  more 
complex  site.  The  site  was  more  challenging  in  at  least  two  dimensions.  First,  the  area  had  been 
used  for  a  wider  variety  of  munitions-related  activities  than  the  previous  sites,  including  not  only 
bombing  ranges  but  also  firing  and  training  ranges.  Second,  the  site  was  used  for  a  much  wider 
variety  of  non-munitions-related  activities,  especially  including  mining  exploration.  As  a  result 
of  both  these  factors,  the  Former  Camp  Beale  site  presented  a  much  wider  range  of  potential 
features.  Nevertheless,  the  overall  objectives  of  the  demonstration  were  met: 

•  LiDAR  and  orthophotos  were  used  to  identify  potential  bombing  targets  and 
firing  ranges  with  a  high  level  of  confidence. 

•  LiDAR  and  orthophotos  were  used  to  correct  what  appeared  to  be  an  erroneous 
target  location  in  the  initial  CSM. 

•  LiDAR  and  orthophoto  data  were  used  to  produce  lists  and  locations  of 
ambiguous  features  for  further  investigation. 

•  LiDAR  and  orthophoto  data  provided  information  on  topography  and  vegetation 
that  was  used  to  plan  magnetometry  and  EMI  transects. 

The  primary  difference  between  the  Former  Camp  Beale  site  and  first  phase  sites  was  that  at 
Former  Camp  Beale  there  were  a  larger  number  of  features  whose  origins  could  not  be 
determined  using  the  LiDAR  and  orthophoto  data  alone.  The  history  of  mining  exploration  at 
the  site  was  particularly  problematic  since  this  activity  produced  depressions  that  could  not 
easily  be  distinguished  from  potential  craters.  However,  the  origin  of  most  or  all  of  these 
features  could  be  resolved  with  field  investigation  using  handheld  magnetometry. 

These  results  emphasize  the  appropriate  use  of  LiDAR  and  orthophotos  at  the  beginning  of  the 
site  investigation  process,  and  the  importance  of  following  the  use  of  LiDAR  and  orthophotos 
with  technologies  such  as  magnetometry  and  EMI  that  directly  detect  munitions  components. 

6.6  END-USER  ISSUES 

End  users  were  not  directly  involved  in  the  demonstration  of  LiDAR  and  orthophotos.  However, 
end  users  are  represented  on  ESTCP’s  WAA  Advisory  Group  for  the  WAA  Pilot  Program. 
Further  information  on  end-user  issues  is  presented  in  the  Final  Report  for  the  WAA  Pilot 
Program  as  a  whole. 

6.7  APPROACH  TO  REGULATORY  COMPLIANCE  AND  ACCEPTANCE 

The  only  specific  approval  needed  to  carry  out  the  demonstration  was  flight  clearance  over  parts 
of  Beale  Air  Force  Base,  which  is  adjacent  to  the  Former  Camp  Beale  demonstration  site. 
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Obtaining  this  flight  clearance  required  approximately  one  month.  Generally,  no  special  licenses 
or  permits  are  needed  to  carry  out  surveys  using  LiDAR  and  orthophotos. 


Further  interactions  with  government  and  quasi-government  validation  programs  are  discussed  in 
the  Final  Report  for  the  WAA  Pilot  Program  as  a  whole. 
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